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Computers provide particularly powerfiil environments 
in which to examine individual differences in cognitiye processing 
and learning outcomes. The computer's capacity to collect and record 
response protocols facilitates detailed process analysis. Such 
analyses contribute to increased understanding of how individuals 
differ in their ability to profit from instruction and learn most 
efficiently. The cognitive engag students use are 

critical inthe computer environment as well as in a variety of other 
learning environments. The cognitive engage^ 

more or less successful learners in one computer environment were 
investigated in a group of 48 California junior high school students 
using a computer problem-soiving game called Wumpus. Results 
indicated that protocols of more and less successful students were 
distinguishableby the spontaneous use of self -regulated learning 
processes* Successful students were able to shift levels of cognitiye 
engagement ihresponse to computer stimuli and feedback^ implications 
for ^ the measurement of self -regulatory processes in computer learning 
environments were discussed. Two other research projects r_t^ 
Thinking and Curriculum Innovation (STAC! ) project and the Structural 
Thinking Experimental Learning Laboratory with Animation (STELLA), 
are also described. (Author/GDC) 
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Computer Learning Environments and the 

Study of Individual Differences iri Sieif -Regulation 
Ellen B. Mandihach 
Educational Testing Service, Princeton, N. J. 

^ ??™P?^®??_P^9'^^^®_P??^^^^^^^^y powerful environments iti which 
to examine individaai_ differences in cognitive pi-dcessihg and 
learning 2??????^^ computer's capacity to collect and record 

response protocols facilitates detailed procesis analysis. Such 
analyses contribute to increased_anderstanding of how individuals 
differ in their ability to profit from instruction and learn most 
efficiently. The cognitive engagement processes students use are 
critical in computer as_weii_as a variety of learning environments. 
The study reported investigated the cognitive engagement processes 
used by more or less successful 'learners in one computer 
enviroTunent . Results indicated that protocols of more and less 
successsful students were distinguishable by the- spontaneous use of 
self- regulated learning processes. _ Successful students were able 
to shift levels of cognitive engagement in response to computer 
stimuli and feedback. Implications for the measurement of self- 
regulatory processes in computer learning environments is 
examined. 

How to improve individual learning is an issue underlying the 
disciplines of differential psychology, instructional psychology, 
and applied cognitive cognitive science. Differential and 
instructional psychology hold that students bring to an 
instructional situation certain psychological characteristics, 
called aptitudes, that influence lieaming. Much research shows 
that learners with different aptitude profiles learn better under 
different instructional methods (Crdtibach & Snow, 1977). One 
explanation is that through demand characteristics, instruction 
stimulates or fails to stimulate certain cognitive processes in 
learners. Just vThich cognitive processes are stimulated during 
learning and how this occurs in computer-assisted instruction (CAI) 
is an important issue for applied cognitive science (Brown & 
Burton, 1978; Burton & Brown, 1979). 
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A related issue for educational psychology is the manner by 
which various student aptitudes interact with learner cognitive 
processes and instructional methods. Much media attention has 
focused on the computer as ah effective medium of instruction. 
Moreover, isdrae computer learning environments provide excellent 
opportunities iii which to examine students' acquisition and 
transfer of higher-order cognitive skills. Of particular interest 
in the present line of research is the extent to which students are 
able to acquire general probliem solving and self -regulatory skills 
and transfer them to related domains. 
Metacognitive Skills 

Examination of metacognitive and self -regulatory skills has 
gained increasing prominance in reisearch on individual 
differences, cognitive processing, and learning (e.g.. Brown, 1978 j 
Como, 1986a, 1986b; Corno & Mandinach. 1983; Flavell, 1976, 1979; 
Gitbmer & Glaser, in press). Metacognition and self -regulation are 
two similar psychological constructs that are procedural in nature 
and thought to be applicable across domains. Both constructs, as 
defined, are composites of behavior, rather than specific skills. 
Metacognition generall}'- is defined as an individual's knowledge 
about one's own cognitive processes (Flavell, 1976). it refers to 
the active monitoring and consequent regulation and orchestration 
of cognitive processes. 

Brown (1978) notes that there are several general 
metacognitive skills characteristic of efficient thought and of 
particular importance because they are "transsituatibnal" . 
Prediction is the first component skill. Learners must be able to 
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predict effects and oatcomes of actidtiis prior to making a iresjpbnse. 
Second, learners nrast be able to plan efficient solutions paths. 
Third, learners must monitor ongoing activity to determine how they 
ares doing, if the response makes sense, and if there are any errors 
or inconsistencies. According to Brown, there are other iskills, 
less well defined, that serve controlling and coordinating 
functions in the learning process, 

Metacognition requires active involvement on the part of the 
learners. It also requires that learners exhibit awareness not 
only of the demands of the particular task or learning environment, 
but more imjportantly of their own capabilities and performances. 
Thus, learners must be able to evaluate and supervise their own 
cognitive behavior through the use of self -interrogation. 
Correspondingly, they must be able to adapt their pierformance in 
accord with task demands. Such flexibility requires learners to 
exhibit sensitivity to task characteristics (Frank, Vye, Auble, 
Mezynski, Perfetto, Bransfbrd, Stein, & Littlefield, 1982). 

Self -regulated learning has been defined as a student's active 
acquisition and transformation of instructional material (Como S 
Mandinach, 1983). The construct consists of two compionent sets of 
processes information acquisition processes and informatibri 
transformation processes. Building on cognitive learning theories ^ 
a first component consisting of information acquisition processes, 
includes receiving stimuli, tracking information, and self- 
reinforcemeht. These processes are seen as metacognitive when they 
regulate the second component of information transformation. 
Important transformation processes include discriminating relevant 



from irrelevant ihformatiohi connecting hew information with prior 
knowledge or skills, and planning particular performance routines. 
Cognitive theories emphasize the significance of the various 
acquisition and transformation processes in the learning and 
maintenance of complex knowledge and skills (e.g., Anderson & 
Bower, 1973; Flavell, 1981; Kintsch & vanDi^k, 1978; Posner & 
Boies, 1971; Reder, 1979). 

Self- regulated learning is viewed as a normative ideal that 
few students use consistently. It is neither appropriate for nor 
encouraged by all classroom tasks. Rather, students are 
hypothesized to alternate between different forms of cognitive 
engagement or variations on self -regulated learning (four forms 
have been conceptualized -- self -regulation, task- focus, resource 
management, and recipience), both between and within different task 
situations (Corno & Mandinach, 1983). Moreovpr, the impetus of 
shifts among the variations may often be task demands and/or 
features of instruction. Learning can become less self -regulated 
when some self -regulation processes are assumeed by teachers, 
peers, or characteristics of instructibhal materials. 

Brown and DeLoache (1978) note that novices and children may 
fail at a task due to a lack of ability and also because they do 
not regulate their actions or have a conscious awareness of them. 
If a learner is not familiar with a given task, that lack of 
knowledge will be manifested in inadequate monitoring and activity 
selection. Familiarity is experiehtially based. With time, 
learners bring to bear such metacognitive skills as predicting, 
monitoring, testing, and controlling one's actions toward a goal, 
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As Brown and DeLdache note, "Although absolute hovicess tend riot to 
incorporate effective metacdgnitive activities into their initial 
attempts to solve problemis, it is libt simply the case that experts 
do and novices do not engage in effective self -regulation" (p. 14). 
Simon and Simon (1978) found that experts exhibited less observable 
self- interrogation than did novices when solving physics problems. 
It is possible that expert performance is so automatic, in some 
instances, that overt self -regulation is no longer necessary. 
Thus, Brown and DeLoache l^pothesize that: (a) novices do not 
engage in self -regulation; (b) self -regulation becrmes increasingly 
important as learners gain experience and knowledge; and (c) 
experts • performances routines may become so automatic tliat self - 
regulation is riot required to such a great extent. 

Two studies that examine self -regulated learning are reported 
here. The first is part of a larger, laboratory-based study of 
which only a subset is described. This research examined the role 
of strategic planning knowledge and self-regulation in learning an 
intellectual computer game. The study of self -regulation was a 
major component of this work. The second study is part of an 
ongoing research project in which self -resgulatibn is only brie of 
several targeted variables. This research fbcustes on the impact of 
computers on learning and transfer and is classrbbm-based. General 
implications for the meajsuremetit of self -regulated learnirig in 
different computer learning environments then are explored. 



5 

6 



Th e Wuitipus Stud y 

Method 

Sample selection . The sample consisted of 48 seventh and 
eighth grade volunteers from a San Francisco Bay area Junior high 
school. Sample selection was based on scores from standardized 
achievement tests (measures of G^) and scores from a battery of 
group-administered ability tests (measures of Gf) , and C3f 

represent intellectual ability distinctions found to be important 
in academic tasks (Cattell, 1971; Snow, 1980). A general 
intellectual ability (G) composite was formed for each student by 
summing over the and Gf indices. 

WumpUs . The study Used ah instructional version of a computer 
problem solving game called Wumpus. Wumpus is a "hunt the monster" 
game in which the student is a hunter whose goal is to track down 
and kill a n^thical creature called a Wumpus, while avoiding 
several hazards that impede safe movement through a warren of 20 
interconnected caves. Task analyses (Mahdinach, 1984) 
demonstrated that successful performance requires the deliberate 
and efficient application of the strategic planning and logical 
reasoning processes that define self -regulated learning. 

Procedure . Four identical instruction-practice sessions were 
designed. Each session contained 12 practice games and 12 
instructional example sets. Students received a game and thetl a 
set of instructional examples. Games were played with minimal 
external assistance. Following the conclusion of a game, students 
received ah example set, while the experimenter added probes 
focused on appropriate performance strategies. Alternation of 
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games and instractional examples peinnitted the assessment of 
performance variations in two different phases of learhihg- 
ihstruction and practice. Data reported here, from the gaining 
phase, are a subset from a larger study (see Maiidinach, i984a, 
1984b; Mandinach & Gorno, 1985 for details). 
Results and Discussion 

The primary Wumpus performance measure was the percentage of 
successful games played by each student. Percentage differences 
are evaluated in terms of effect size coefficients (ES) which 
reflect the magnitude of differences between groups in terms of 
standard deviation units but d^ not depend on sample size (Glass, 
1978). High ability students proved more successful on Wtimpus thati 
low ability students (M - .39 for the high group and M - .20 for 
the low group, ES - 1.42). 

Self-regulated leartiihg and strategic planning were measured 
by an index called "error avoidance." This was the percentage of 
unnecessary risks successfully avoided during the beginning part of 
a game. A high score on error avoidance reflected a student's 
tendency daring play to consider alternative solutions and plan a 
logical sequence of moves. High ability students (H * .54 for 
hlgjis vs. M * .31 for lows) displayed more self -regulated learning. 
Error avoidance also correlated with success on Wumpus r(47) - ,66, 
£ < .001, suggesting that those who explored more alternative 
solution paths before asstming an initial risk also were mbire 
successful. 

In addition to the error avoidnace measure of self -regulated 
learning, students' verbal protocols, response patterns, and study 
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aids were examined along with experimenter notes for evidence of 
the cognitive processes that defined the four hjnpotheticai forms of 
cognitive engagement. Games and instructional example sets were 
scored by defining specific cognitive skills and behavior that 
reflected connecting, alertness, selectivity, planning, and 
monitoring. Evidence of these skills was noted for each occurrence 
in a game or example set and categorized according to the primary 
and secondary forms of engagement used across sessions. 
Reliability of these categorizations was assessed as percent 
agreement between two independent raters on a subset of protocols. 
Agreement was ±00% for the primaxry form of engagement coded and 80% 
frr secondary forms . 

Table 1 presents the gaming data for all students and is 
broken do\m by ability level. Seventy-three percent of the 
students sampled were observed to adopt and maintain the same form 
of cognitive engagement throughout the gaming sessions. Only 27% 
of the students shifted to a different form of cognitive engagement 
at least once across occasions. Among the form of cognitive 
engagement noted, self -regulated learning was the most frequent 
(27%). Resource mahagmeht was noted the least frequently (12%) but 
was the form most often used in combination "'rith other engagement 
forms. Like resource management, task- focused learning tended to 
be used in cdmbitiatibn with other engagement variations, but less 
frequently overall. 

The two measures of self -regulated learning (i.e., the error 
avoidance index arid the categorizations made here) were correlated 
in these data. Students whose ddmihaht rating was self -regulation 
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(M - .68) also had higher scores oh error avoidance than the 
remainder of the sainple (M « .33), t(46) - 5.14, £ < .001. Such a 
relationship provides evidence suppdrtirig the construct validity of 
these measures. 

Students who used self -regulated learning were more successful 
in Wumpus that those yAio used other forms of cognitive engagement. 
Self-regulated learners had the highest percetitageis of success (M - 
.50) followed by students who combined task-focused and self- 
regulated learning (M - .38). The least successful istudents 
combined recipiesnt lesaming with resource management (M - .15). 

The results of this study should be seen in relation to other 
research investigating the cognitive processes that underlie 
effective perfbrmahce bh complex learning tasks. Consistent with 
data described by Pressley and Levin (1983a, i983b), we find 
empirical evidence that students actively engage in a relatively 
small iset of acquisition and trahsfbrmatibnal processes to aid them 
in task performance. The Wumpus prbtbcbls indicated that 
performance was better when students used the underlying processes 
that define self -regulated learning. The transcripts of less 
successful students were marked by the relative absence of these 
processes . 

The Wumpus computer gaine was selected ^ -fti part, because 
rational task analyses suggested that effective perfbrmahces 
required just such cognitive activity as was observed in the data. 
The game also provided an attetnpt to induce the use of key self- 
regulation activity in students who were riot usirig such prbcesses 
on their own, and to compare the performarice butcbmes of students 
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trained with those of spdhtaheous users (see Mahdihach, 1984a, 
1984b, 1987 for details of instructional results). 

The data also support the hbtibti that differences in student 
aptitudes may be related to differences in the task approach or 
cognitiro^e engagement strategies that systematically relate to 
performance outcomes. Students of different aptitudes also 
displayed difference levels of self -regulated learning and 
different forms of cognitive engagement during interaction with the 
computer. Specifically, high ability students showed more evidence 
of self-regulated learning and tended to shift among the different 
hjrpothetical fbznns of cognitive engagement as they played. These 
stuiehts shift esd in response to computer feedback on their 
performance. In essences, they were more "response sensitive" than 
other students to incoming information. The issue of what features 
of learning tasks are most likely to trigger such strate^ shifts 
is an interesting one to pursue. 

Low ability students were found to use different forms of 
cognitive engagement in this study as well, but the primary form of 
engagement observed was recipient learning. Some of the low 
ability student appeared to shift from recipient learning to other 
forms of engagement as their experience with the games progresssed - 
again indicating a kind of response sensitivity. The manner by 
which learners of different ability levels react to cues in the 
computer learning envirotiffient has only begun to be investigated 
(e.g., Mandinach & Fisher, 1985; Webb, 1984). 
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In sum, patterns of cognitive enagement were found to differ 
between studexits of high and low ability. Some students used seif- 
regulated learning in Wumpus but others did not; these differences 
were related systematically to ability and performance outcomes; 
Students who used self -regulated learning and the other higher 
forms of engagement (task- focused learning, resource mcnagement) 
generally were more able and more successful on Wumpus i 

The STACI Project 
The Systems Thinking and Curriculum Innovation (STACI) project 
is a two-year research project conducted by Educational Testing 
Service under the aurpices of the Educational Technology Center at 
the Harvard Graduate School of Education. The project is intended 
to examine the cognitive demands arid consequences of learning from 
a systems thinking approach to instruction and from using 
simulation modeling software. 

The purpose of the study is to test the potentials and effects 
of using the systems approach in existing secondary school 
curricula to teach content-specific knowledge as well as general 
problem solving skills (see Mandinach fe Thorpe, in press for 
details). The stu^ aiso examines the effectiveness of using 
STELIA (Structural Thinking Experimental Learning Laboratory with 
Animation; Richmond, 1985) as a tool by which to teach system 
dynamics and content knowledge. The research focuses on (a) the 
learning outcomes and transfer that result from using such an 
approach and software in classroom settings, and (b) the general 
effects of teaching with the technology. 
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The study is beitig conducted at a high school in southern 
Vermont in which four teachers are usitig systems thinking in their 
courses. The course content areas include general physical 
science, biology, chemistry, and an experimental course entitled 
War and Revolution. These four teachers, trained to use STELLA and 
system dynamics, are using systems models and illustrating them on 
the computer. 

The intent of the research project is to examine the extent to 
VThich students acquire higher-order cognitive skills (specifically 
seif-reguiation and general problem solving skills) through 
interaction with a curriculum infused with systems thinking 
concepts and subsequently generalize knowledge and skills to 
problem solving tasks in other substantive areas. Comparisons are 
being drawn between traditionaiiy taught courses and those that use 
the systems approach and STELLA. Furthermore, the two-year 
duration of the research enables the examination of skill and 
knowledge transfer across content areas as students are exposed to 
several courses that use the systems approach. 

Measurement of Self-Reguiated Learning 
The STACI Project provides an opportunity to examine seif- 
regulated learning in classroom settings to which substantial 
computer and thinking skills cbmpbhehts have been added. Howevesr, 
real -world constraints of classroom data collection require a 
substantially different methbdblb^ than that used in the Wumpus 
study. The purpose of discussing the STACI Project is tb highlight 
some of the differences between classrbbm and labbratory-based 
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studies that attempt to exasaitie higher-order cognitive processes 
such as self- regulated learning. 

Laboratory studies generally allow for a one-to-one 
interaction between the researcher and the learner. Thus, there is 
greater opportunity to record systematically incidents of the 
targeted behavior. For example, in the Wumpus stu^, data were 
collected from a variety of sources and triangulated to provide a 
detailed picture of students' cognitive processing and use of self- 
regulatioh. These sources included interviews, think-aloud 
prbtbcbls, teachbacks, experimenter's notess, tape recordings of 
experimental sessions, keystroke logs, and tasks or indices (e.g, 
the error avoidance variable) specifically designed to measure 
higher-order cognitive skills. Each of these sources, separately 
and in cbmblnLtibhi prbvided valuable evidence of cognitive 
engagemetic. 

Such an ^^xamitlatibn bf the learner is hot possible in 

a classroom setting because extratiebus variables interfere with 
precise measurement. Often times learning is the outcome bf group 
interaction rather than bnie student's cbtlgitive prbcessing. Thus^ 
methodology must be adapted according tb the cbhstraints imposed by 
the classroom. 

The main techniques used in the STACI Project tb examine self- 
regulated learning are instruments specifically designed tb assess 
students' cognitive engagement. The instruments have been designed 
to measure the various component processes that comprise self- 
regulated learning and metacognition (i.e., planning, monitoring, 
alertness, selectivity, connecting). How students perform on these 
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exercises should indicate the extent to which they are able to 
exhibit forms of cognitive engagement on isolated taisks. 

The curricula implemented in the Project's experimental 
classes (see Mandinach & Thorpe, in press) contain numerous 
opportunities for students to erfiibit self- regulation. These 
learning activities will be examined with various methodologies 
(e.g., observation, specific exercises, and interviews) appropriate 
to classroom situations. Acquisition and the extent to vThich the 
targeted skills are exhibited will be assessed within classes. 

The design of the study allows for the examination of transfer 
across cbriterit areas. Because students are required to take three 
years of high school science ^ we will be able to trace the 
development and use of the targeted skills and knowledge across 
science courses. We also will be able to examine the differential 
effects of the experimental and traditional instructional 
treatments dii the acquisition of higher-order skills and content 
knowledge. Transfer of these inetacbgnitive skills across domains 
will be examined as students take additional courses that are 
taught with the systems thinking approach. Thus^ it will be 
possible to examine level of cognitive processing and the extent to 
which student use and transfer self-regulation across learning 
situations. 

In sum, attending to, organizing, and using feedback such as 
the computer supplies was a critical element in success in the 
Wumpus study and should influence performance in the STACI Project 
as well. Sensitivity to feedback has been viewed as an importnat 
component of cognitive engagement. The Wumpus study demonstrated 
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that not all students are alike in the extent to which they can 
adapt to taisk demands, respond to feedback^ and know when different 
levels of cognitive engagement are apprbjpriate. The STACI Project 
intends to followup oti this finding. The flexible use of such 
higher cognitive skills across tasks is a general goal for ali 
education (Cortio & Mandinach» 1983; Paliticsar & Brown, 1984; Snow & 
tohman, ±984). Subsequent research will attempt to illuminate the 
roles of self -regulated leaining and response sensitivity in 
various classes of learning activities. 

Computers provide powerfrul learning, instructional, and 
research environments. The computer can play a critical role as 
the medium of instruction, as in both the Wumpus study and STACI 
Project, and as a research tool, especiaiiy in the Wumpus study. 
The computer's capacity to present stimuli and to collect and 
record response protocols facilitates analysis of learners' 
cognitive processing. However, the extent to which irtdepth 
analyses of individual learners can be accomplished depends on the 
design of the research, level of analysis (e.g., classroom, small 
group, individual learner), and the particular varia'Dles targeted 
for examination. Methodologies must be designed to capture the 
nuances of each research setting in order to capitalize on the 
power of the computer as both an instructional medium and research 
tool. 
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Table 1: Cogntttve Engagement In Gaaas^ (N » 48) 



Engagement Form High Abtlity ^\<t-Ability Total 





n 


% 


n 




ft 


%^ 


Self -regulation (SR) 


12 


50 ;G 


i 


4.2 


13 


27.1 


lasK rocus (TF) 


4 


16.7 


3 


12.5 


7 


14.6 


Resource management (RM) 


2 


8.3 


it 


16.7 


6 


12,5 


Recipience (R) 


0 


0 


9 


37;5 


9 


18.8 


TF 6c RH 


1 


4.2 


1 


4.2 


2 


4.2 


RM 5c SR 


2 


8.3 


1 


4.2 


3 


6.3 


TF & SR 


1 


4.2 


1 


4.2 


2 


4.2 


R & RM 


1 


4.2 


3 


12.5 


4 


8.3 


RM, TF. & R 


1 


4.2 


1 


4.2 


2 


4.2 



is 
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